====================== 
SUMMARY: 
====================== 

 - ESP-r compiles correctly on Linux, Cygwin, MinGW, Solaris and OSX 
 - GCC 4.1.2 remains the reference compiler
 - We are no longer thoroughly testing GCC3
 - GCC 4.2.3 builds are not as robust
 - GCC 4.3 and 4.4 have not been tested
 - Some uncertainty remains in Intel builds, as well as builds on OSX 
  
Recommendations: 
 - Proceed with release 11.7 
 - The ESP-r 'reference' platform is Linux / GCC 4.1.2 
 - Discontinue support for gcc 3.4.6 as soon as stable versions of 
   GCC4 are available on all platforms.
 - SUN CC builds should continue to be designated as stable
 - GCC 4.2+ and Intel builds should be used with caution
 - Compiler optimization is not recommended based on experience
   from the previous release cycle 
 - OSX / GCC 3.4.6 and MinGW / GCC 3.4.6 builds should be used with 
   caution

Detailed results follow. 



====================== 
Static Analysis: 
====================== 

The Forcheck static analyzer warns of no additional coding errors in the
current version of pre-release-patches. The total number of errors,
warnings and informational numbers as declined from the last release.



====================== 
Regression testing: 
====================== 

Changes since the last revision have introduced some numerical
differences into the results. The numerical differences were anticipated
by developers, and no unintended effects were observed during testing. 
One change has also increased overall efficiency, reducing simulation run-times
significantly. 

Two revisions introduced numerical differences into the results: 

 r4129: This commit patched a bug in ish that caused ish to incorrectly
        compute a surface's azimuth when the first edge of the surface
        was not in the horizontal plane. Most models generated with prj are 
        unaffected, as prj always specifies the lower horizontal surface
        first when defining conventional rectangular geometry. Still, this
        change will correct previously erroneous shading computations for
        complex, unconventional geometry, or for geometry generated by 
        third-party applications. A model has been added to the test_suite
        (obs_4_eachface) to account for this in future testing. The addition
        of this model resulted in anticipated numerical differences. 

        This commit also re-enabled code that reports AC system performance
        as computed by the ideal HVAC models. This resulted in anticipated
        numerical differences in the following idealized_hvac models:
        ccht_gcep_summer
        ccht_gshp.H4_summer
        ccht_gshp.H4_summer_MooreModel
        ccht_gshp.HS_summer
        ccht_gshp.SL_summer
        ccht_gshp.V1_summer

 r4092: This commit introduces the complex fenestration construction (CFC)
        and a new test_suite model (complex_fenestration) to support the 
        facility. Anticipated numerical differences were observed from the
        addition of this model. 
        
One revision resulted in a significant increase in efficiency:

 r4233: This commit removes a trace loop, which is executed 6720 times per
        time step, from the main code sequence and only executes the loop if 
        trace is on. 

====================== 
Portability testing: 
====================== 
The release candidate can be successfully installed on the following
platform, compiler and graphics library combinations. We've also added
support for the Intel compiler suite on linux. Jon Hand has been
experimenting with support for 64-bit architecture, but this work has
not yet been generalized for inclusion in the public release. 

We've successfully installed ESP-r using the following 32-bit platform,
compiler and graphics library combinations: 

          Support for ESP-r on various compiler, platform 
                and graphics library combinations. 
    ===========================================================================================
    Compiler     Library    Linux   Cygwin   MinGW   Sun-Sparc(64bit)   Sun-Intel(32bit)   OSX 
    =========================================================================================== 
    GCC 3.4.X    X11                                      O                                 O 
    SUN CC       X11                                      O                     O
                 noX                                                            O
    GCC 3.4.4    X11                  O 
                 noX                  O 
                 GTK                  O
    GCC 3.4.5    noX          O                *                                            O
                 X11          O                                                             O
                 GTK          O                *                                            O
    GCC 3.4.6    X11          O 
                 GTK          O 
                 noX          O 
    INTEL        noX          O 
                 X11          O 
                 GTK          O 
    GCC 4.1.2    X11          O 
    GCC 4.2.3    X11          O
    GCC 4.3.0    X11                                                                        O
                 noX                                                                        O
                 GTK                                                                        O
    GCC 4.3.2    X11                  O        O 
    ------------------------------------------------------------------------------------------
    Notes: 
    O: Installs correctly. 
    *: MinGW only installs correctly to absolute DOS paths 
    (i.e. C:/ESRU) 
    ==========================================================================================

We've also undertaken a comprehensive comparison of the numerical
results produced by ESP-r on various platforms. 

We've designated GCC 4.1.2 /Linux builds as the reference version of ESP-r.
Most ESP-r development and automated testing is currently based on this
platform, and bps's numerical predictions on this platform receive more
scrutiny than any other. Although ESP-r can be compiled with GCC 4.3 and 4.4, 
we have not thoroughly tested it with these compilers. Most Linux
distributions now come with GCC 4.3.x, but we still recommend GCC 4.1.2
as we've seen instability between GCC 4.2/4.3 releases in the last release
cycle, particularly with gFortran.  

The following chart summarizes the significant observed differences
between the GCC 4.1.2 / Linux platform and other platform and compiler
combinations. The "Maximum difference" reflects the maximum difference
observed in the test cases, excluding exceptional test cases that are
discussed in section 'Problematic test cases', below..

      Comparison of results from GCC4.1.2-compiled versions 
      (X11, Linux) with builds from other platform/compilers. 
    ============================================================= 
    Arch.     Platform    Compiler      Maximum difference (%) 
    ------------------------------------------------------------- 
    IA-32     CYGWIN      GCC 4.1.2            0.63 
              LINUX       GCC 4.1.2            (REF)
              SOLARIS     Sun CC/f90           5.28 (note 2)
              LINUX       Intel -O0            0.75  
    PowerPC   OSX         GCC 3.4              3.41 
    Sparc     SOLARIS     Sun CC/f90           2.61 
    ------------------------------------------------------------- 
    NOTES: 
      1. Some test cases produced numerical exceptions and 
         floating point errors. See "Problematic Test Cases".
      2. Two test cases caused a numerical exception. See below. 
      3. Testing on MinGW was not carried out due to an issue
         with DOS path management in ESP-r. 
    ============================================================= 
    
=========================== 
Problematic Test Cases 
=========================== 

 - esru_benchmark_model / bld_basic_af2_summer 
                        / bld_basic_af2_winter 
   ------------------------------------------- 
   We've previously observed that this coarse-timestep air flow 
   network test case produces significant differences in g77 and 
   gFortran builds. The long, half-hour timesteps cause small 
   differences in the numerical computations to be exaggerated in 
   the aggregated output --- increasing time resolution vastly improves 
   agreement between the compilers. 

   The Linux / Intel, Solaris / SUN CC-f90, and OSX / g77 platforms all 
   exhibit the same sensitivity. They produce differing results at 
   short-timesteps, but agreement with Linux / gFortran improves at higher 
   time-resolutions. 

   Since we've observed that each compiler combination produces
   dissimilar results and that the high-resolution version is more useful, 
   this test case has little value. For this reason, I recommend we 
   delete it. 
   
   
-  bat_life_no_control / h2-ctrl
   ------------------------------------------------- 
   This test case consistently produces large errors across all
   platforms except Linux GCC 4.1.2 and Cygwin GCC 4.1.2 and will be 
   investigated during the next release cycle.
   
   
-  bld_hc_ISO15099 / HC
                   / HC_no-ISO
   ------------------------------------------------- 
   These test cases consistently produce large errors across all platforms
   and will be investigated during the next release cycle. 
   
   
-  cellular_offices / office_operations
   ------------------------------------------------- 
   This test case consistently produces significant (~5%) errors across all
   platforms and will be investigated during the next release cycle. 
   
   
-  idealized_hvac / ashp_cool_auto_conv_winter
                  / ashp_cool_cont_conv_winter
   -------------------------------------------------   
   While they run correctly in every other compiler, these test cases appear
   to produce numerical exceptions in the SOLARIS / CC f90 platform, 
   reporting temperature values of -99.9(oC). This will be investigated in 
   the next release cycle. 
